Image sensors, including those used for video applications, have pixel cells for detecting light. Typically, each pixel cell integrates (e.g., accumulates) an amount of charge representing an intensity of detected light. As a reference to photosensitive film, pixel cells are commonly said to be “exposed” for the duration that they integrate such charge. After integration, each pixel cell reads out an analog pixel signal representing the amount of integrated charge and, in turn, thereby representing the intensity of the detected light. The pixel signal is used to generate digital pixel data, i.e., a digital pixel value, representing the intensity of the detected light in a digital format. The pixel signals generated by a collection of pixel cells, which are typically arranged as rows and columns of an array, are used to generate pixel data representing an image captured by the image sensor.
FIG. 1 is a circuit diagram of a conventional four-transistor CMOS pixel cell 170. Prior to integration of the pixel cell 170, the reset transistor 142 and charge transfer transistor 128 are simultaneously switched on (by applying a reset signal RST and charge transfer signal TX to their respective gate terminals) to connect the photosensor 124 to a voltage source 130 (e.g., a source of a VDD voltage) and drain the photosensor 124 of previously integrated charge. During integration, which begins in response to switching off both the reset transistor 142 and charge transfer transistor 128, light strikes the photosensor 124 to integrate charge. For example, light may strike the pn-junction of a photodiode and thereby cause electrons to accumulate within the n-type region. After integration and in response to a charge transfer signal TX, the charge transfer transistor 128 gates the integrated charge from the photosensor 124 to a storage node 126 (e.g., to a floating diffusion region). In this example, the transfer of integrated charge marks the end of integration and the beginning of the pixel signal readout.
The transferred charge biases the gate of a source follower transistor 132. A first terminal of the source follower transistor 132 is connected to the voltage source 130. A second terminal, which transmits an image signal Vsig indicating the amount of integrated charge stored by the storage node 126 or a reset signal Vrst indicating the reset voltage stored at the storage node 126, is connected to a column line 136. In response to a row select signal ROW, a row select transistor 134 gates the image signal Vsig (or reset signal Vrst) to the column line 136. The storage node 126 is reset by activating the reset transistor 142 to connect the voltage source 130 to the storage node 126.
An example of a pixel signal readout operation is described with reference to FIG. 2, which is a block diagram illustrating a conventional CMOS image sensor 101. As shown, the image sensor 101 includes a pixel array 30 having rows 30R and columns 30C of pixel cells 170. The pixel array 30 is connected to a row selector 42. Under the direction of the timing and control circuit 44, the row selector 42 causes each pixel cell 170 of a selected row 30R to transmit at least one output signal Vout, via its respective column line 136, to a sample-and-hold circuit 45. A column bus 43 then selectively passes the output signals Vout to a system of components for generating pixel data.
In this example, each pixel cell 170 transmits two output signals Vout, a reset signal Vrst and the image signal Vsig, to the sample and hold circuit 45. The reset signal Vrst and image signal Vsig of each pixel cell 170 within the selected row 30R are stored by respective capacitor in the sample and hold circuit 45. After the sample and hold circuit 45 stores the reset signal Vrst and image Vsig signal for a respective pixel cell 170, those signals Vrst, Vsig are converted to a differential signal (Vrst−Vsig) by a differential amplifier 46. The differential signal (Vrst−Vsig) is digitized by an analog-to-digital converter (ADC) 47, and the digital pixel data is input as an input signal SIGin to an image processing chain contained within an image processor 48.
Image capture is performed as part of a global shutter or rolling shutter operation. In a global shutter operation, all of the pixel cells 170 within the pixel array 30 would be integrated at the same time. In a rolling shutter operation, the pixel cells 170 of the pixel array 30R would be integrated on a row-by-row basis. Thus, in a rolling shutter operation, the start time and end time of integration is different for each row 30R of pixel cells 170. However, in both a global shutter operation and rolling shutter operation, the duration of integration would typically be equal for each row 30R of pixel cells 170; and the duration of the pixel signal readout operation would typically be equal for each row 30R of pixel cells 170.
Regardless of whether a global shutter operation or rolling shutter operation is employed, the pixel signals of the pixel cells 170 are read out on a row-by-row basis. For example, the image signals Vsig and reset signals Vrst of all pixel cells 170 within a selected row 30R may be simultaneously stored by respective capacitors of the sample-and-hold circuit 45 and then, under the direction of the control bus 43, passed to the differential amplifier 46. Such a readout would be repeated for each row 30R of the pixel array 30.
The image sensor 101 may write the pixel data into a buffer used by the image processor 48. In such instances, the buffer would typically store the pixel data of less than all of the rows 30R of pixel cells 170 of the image sensor 101, and overwrite the stored pixel data on a first-in/first-overwritten basis, i.e., the first row of pixel data stored would be the first row of pixel data overwritten. The rates of outputting the pixel data from the image sensor 101 and inputting the pixel data to the buffer are typically fixed and equal (i.e., determined by the characteristics of the system). The rate at which the pixel data can be overwritten in the buffer, without losing the stored pixel data that is still being used for processing, is also typically fixed. When new pixel data is written to the buffer, the memory usage of the buffer increases. When the stored pixel data is no longer needed for processing and therefore can be overwritten without losing needed data, the memory usage of the buffer decreases. The rates of these changes are herein referred to as the input rate and discard rate, respectively. The difference of the input rate and discard rate determines whether the memory usage of the buffer is increasing or decreasing at a given moment.
Processing of the pixel data by the image processor 48 may involve, but is not limited to, modifying one row of pixel data (i.e., a line of an image captured by the image sensor 101) based on multiple rows of pixel data. The amount of data required to perform the image processing operation may vary across the image, from pixel to pixel and from row to row. The region of the image at which the largest amount of data is required to perform the transformation gives rise to a fundamental requirement of buffering. In that case, the buffer must store at least the maximum amount of data used to create any one line of output pixel data. This is one example of a “fundamental requirement,” which is the maximum amount of data required simply to perform any processing operation due to spatial considerations. With dewarp, the top or bottom of the image, where distortion is greatest, require the largest amount of input data (and therefore, the largest amount of buffering) in order to perform the transformation; the part of the image nearer the center requires less buffering. In addition to the fundamental requirement, the buffer must also accommodate any increases in memory usage that occur for example where the input rate exceeds the discard rate, or where timing constraints force the loading of part of the data for a second frame of processing before the first frame has finished. Such increases in memory usage can require imaging systems to employ buffer sizes that are substantially larger than the fundamental requirement in order to ensure that the buffer capacity will meet the greatest memory usage of a processing operation. In many systems, however, it is desirable to employ a smaller buffer to conserve die space. Accordingly, embodiments of an imaging method, apparatus, and system are disclosed herein which adjust the output rate of a sensor (and thereby adjust the input rate of a buffer) to decrease the memory usage of a processing operation.